In order to improve the accuracy of flame recognition in video, a flame recognition algorithm based on Codebook was proposed. The algorithm which combined with static and dynamic features of flame was innovatively applied with YUV color space in Codebook background model to detect flame region, and update the background regularly. Firstly, the algorithm extracted frames from video, and used the liner relation between R, G, B component as the color model to get the flame color candidate area. Second, because of the advantage of the YUV color space, the images were transformed from RGB format to YUV format, a flame color dynamic prospect was extracted with background learning and background subtraction by using Codebook background model. At last, Back Propagation (BP) neural network was trained with the features vectors such as flame area change rate, flame area overlap rate and flame centroid displacement. Flame was judged by using the trained BP neural network in video. The recognition accuracy of the proposed algorithm in the complex video scene was above 96% in fixed camera position and direction videos. The experimental results show that compared with three state-of-art detection algorithms, the proposed algorithm has higher accuracy and lower misrecognition rate.